The present invention relates generally to computing systems, and more particularly, to systems and methods for network data distribution.
Significant effort has been devoted to optimizing the operation of large networks such as, for example, wide area networks (WAN""s), the Internet, and the World-Wide-Web (xe2x80x9cthe Webxe2x80x9d). To date, however, those efforts have largely been directed at improving the operation of networks from the perspective of consumers of data. For example, in a network such as the Web, when a Web browser issues a request for a Web page, without network optimization it may be necessary to traverse a large portion of the network in order to reach the one Web server on which the requested page exists. Of course, traversing a large portion of the Web can take a relatively long time. One technique for addressing this limitation has been to distribute the same data items on many servers throughout the network. When a request for a distributed data item is made, it is likely that the data item can be accessed quickly without traversing a large portion of the Web.
Efforts to improve network performance for consumers of data have been very effective. Indeed, the high quality of service provided by modern networks has resulted in a significant increase in network usage. Not surprisingly, increased network usage has resulted in higher levels of network traffic. The resulting increase in network traffic has been so great, in fact, that there now exists a need for means to conserve network resources while continuing to provide the same high level of performance.
Illustrative systems and methods for distributing data while conserving network resources are disclosed herein. In an illustrative embodiment, a first computing system, which may comprise, for example, a digital subscriber line multiplexor (DSLAM) and a proxy server, is communicatively coupled using a digital subscriber line (DSL) to a second computing system, which may comprise a data server. Requests for data such as, for example, a Web page, that are directed to the data server are received first at the proxy server, which forwards the requests to the data server.
The proxy server receives a response from the data server including data items such as, for example, Web pages and data files corresponding to items displayed on the Web pages.
For each data item received, the proxy server determines whether the same data item has previously been received. For example, the proxy server determines whether a Web page with the same file name was previously received. If not, the proxy server stores the data item locally.
If the proxy server determines that a requested data item was previously received at the proxy server from the data server, the proxy server compares the data item to the previously received version of the data item. For example, the proxy server may compare a Web page with a previously stored copy of the Web page with the same name. In comparing the data items, the proxy server determines whether all or portions of the data item have changed. Those portions of the data item that have not changed may be specially identified as virtual-edge-side-assets (VESA). For example, a text portion of a Web page may not have changed and might therefore be a candidate for designation as a VESA. File components designated as VESA""s are saved as separate files and distributed to other proxy servers in the network. The VESA""s are used to reconstruct the data items prior to delivery to the requesting party.
When comparing a data item to a previously stored version of the data item, the proxy server may also determine whether a portion of the data item that was previously designated as a VESA has changed and therefore should no longer be designated as such. In these instances, the proxy server communicates to other proxy servers on which the VESA may have been stored that the particular VESA is no longer valid.
Where a VESA has been identified, the portion of a data item that corresponds to the VESA is removed from the data item and replaced by a reference to the VESA. For example, a text portion of a Web page that is identified as static and therefore designated as a VESA may be removed and replaced with an HTML xe2x80x9cincludexe2x80x9d command directed to the particular VESA. Replacing the data content with a reference to the VESA has the effect of reducing the size of the data item file. The reduced data items are communicated across the network to a proxy server located in proximity to the machine from which the data request originated. Reducing the size of a data item file prior to transmission over the network reduces the bandwidth needed to transmit the file and thereby conserves network resources. Upon receipt of the data item, the second proxy server reconstructs the data item by inserting the body of the VESA""s as appropriate.